Estimation of a person's point-of-gaze (also referred to as point-of-regard) has become an important tool in a variety of applications, including, for example, the study of visual and oculomotor systems, the study of reading behavior, marketing/advertising research, and the control of device user interfaces (e.g., graphical user interfaces) using eye movements.
Methods of estimating a person's point-of-gaze using estimates of an eye's optical axis determined from images of the eye are known. Examples of such methods include: S.-W. Shih, Y.-T. Wu, and J. Liu, “A calibration-free gaze tracking technique,” in Proceedings of 15th Int. Conference on Pattern Recognition 2000, pp. 201-204 (hereinafter, “Shih 2000”); S. W. Shih and J. Liu, “A novel approach to 3-d gaze tracking using stereo cameras,” IEEE Transactions on Systems, Man, and Cybernetics, Part B: Cybernetics, vol. 34, pp. 234-245, February 2004 (hereinafter, “Shih 2004”); E. D. Guestrin and M. Eizenman, “General theory of remote gaze estimation using the pupil center and corneal reflections,” IEEE Transactions on Biomedical Engineering, vol. 53, pp. 1124-1133, June 2006 (hereinafter, “Guestrin 2006”); E. D. Guestrin and M. Eizenman, “Remote point-of-gaze estimation requiring a single-point calibration for applications with infants,” presented at the Proc. of the 2008 Symposium on Eye Tracking Research & Applications, Savannah, Ga., USA, 2008 (hereinafter, “Guestrin 2008”); J. G. Wang and E. Sung, “Study on eye gaze estimation,” IEEE Transactions on Systems, Man, and Cybernetics, Part B: Cybernetics, vol. 32, pp. 332-350, 2002 (hereinafter, “Wang 2002”); J.-G. Wang and E. Sung, “Gaze determination via images of irises,” Image and Vision Computing, vol. 19, pp. 891-911, 2001 (hereinafter, “Wang 2001”); E. Trucco, T. Anderson, M. Razeto, and S. Ivekovic, “Robust correspondenceless 3-d iris location for immersive environments,” in Image analysis and processing—ICIAP 2005. vol. 3617, F. Roli and S. Vitulano, Eds.: Springer Berlin/Heidelberg, 2005, pp. 123-130 (hereinafter, “Trucco 2005”); Amir et al., U.S. Pat. No. 6,578,962, “Calibration-free eye gaze tracking” (hereinafter “Amir”), E. D. Guestrin, “Remote, non-contact gaze estimation with minimal subject cooperation,” PhD Thesis, Department of Electrical and Computer Engineering, University of Toronto, Toronto, ON, Canada, 2010; and D. Model, “A Calibration Free Estimation of the Point of Gaze and Objective Measurement of Ocular Alignment in Adults and Infants”, PhD Thesis, Department of Electrical and Computer Engineering, University of Toronto, Toronto, ON, Canada, 2011, the contents of which are hereby incorporated herein by reference.
As is understood, although the optical axis is an axis of symmetry of the eye-ball, a person's line-of-sight is not directed along the optical axis. Specifically, the human retina does not have a uniform density of photoreceptors. Instead, there exists an area on the retina with a pronounced peak in the density of photoreceptors. This area is known as the fovea and subtends approximately a solid angle of one degree of the visual field. In order to see an object in detail, the eye is oriented such that the image of the object is projected onto the fovea. Thus, line-of-sight may be modeled, for example, as a line that extends from the fovea and through, e.g., the center of curvature of the cornea. This line-of-sight deviates from the optical axis by an angular offset (see, e.g., Guestrin 2006 and Guestrin 2008 noted above, and R. H. S. Carpenter, Movements of the eyes, London, UK: Pion, 1977). This line-of-sight will also be referred to herein as the visual axis; it may also be referred to as the foveal axis, the eye-gaze vector, or the gaze vector.
While the optical axis can be readily estimated from an image of the eye (e.g., based in part on the location of the pupil center in such an image), the visual axis cannot be directly estimated from an image of the eye (e.g., because the fovea is not observable in such images). As such, prior art methods have relied simply on the optical axis of the eye as a proxy for the line-of-sight (see, e.g., Amir), or have estimated the visual axis indirectly by applying estimates of the angular offset between the optical and visual axes to estimates of the optical axis (see, e.g., Guestrin 2008). As will be appreciated, methods that take into account the angular offset between optical and visual axes generally provide more accurate estimates of point-of-gaze than methods that do not take into account this angular offset.
Unfortunately, existing methods of estimating point-of-gaze that take into account the angular offset between optical and visual axes are intolerant to substantial rotations of a person's head relative to an estimation device, or substantial rotations of the estimation device relative to the head, and thus fail to provide accurate estimates of point-of-gaze when such head/device rotations occur. This has, for example, inhibited the implementation of point-of-gaze estimation on mobile devices for which substantial rotations are commonplace.
Accordingly, there is a need for improved point-of-gaze estimation methods and devices.